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Device and method for eye to eye communication over a network 

This invention relates to a device and method for eye to eye communication using 
audio and moving images transmitted over a network. 

Various forms of communication have been developed for transmitting moving images 
and audio between communicators. Commonly known as video conferencing, these 
systems use a network connection, such as ISDN, to transmit video between two 
locations. Most systems use a video monitor to display the people at the distant 
location and have a camera positioned on top of the monitor to capture an image of the 
users in the home location for transmission to the distant location. 

In a common form of video conferencing the users communicate with the people 
pictured on the monitor. This type of communication is unnatural in that the users 
cannot look directly at each other during their communication. This is because the 
camera is located above the monitor instead of matching to the position of the eyes of 
the person displayed on the monitor. If the user in the home position looks at a person 
on the monitor, the user will be looking away from the camera. As a result the image 
displayed on the monitor in the distant location will show the person looking away 
from the camera. This makes it impossible to have natural eye to eye communication.. 

With this present invention the users in both locations will be able to see each other as 
life size images that appear to be in the room with them. In the invention the positions 
of each of the communicators, the cameras and the presentation screens are in a 
configuration that makes it possible for the users can to have eye to eye contact. This 
eye to eye contact makes it easier to establish a sense of presence with people which 
encourages more natural communication. Through this format people can be more 
expressive and draw upon human gestures and expressions to communicate emotions 
and feelings. Instead of having a video conference this invention creates a new form of 
communication that has been coined a 'reality interface conference'. 
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This invention provides a device and method for achieving eye to eye communication 
by matching the position of cameras with the relative position of the eyes of the images 
of the people displayed on a screens. 

The invention comprises a home location connected by a network to one or more 
remote locations with each location having an arrangement of a video camera, a two 
way mirror and a presentation screen. The video camera can be any device that 
captures moving images of the user. The camera may output the moving images in a 
standard video format or a format optimised for a network, such as internet protocol for 
transmission over the internet. The two way mirror can be a semi-transparent surface 
that is comprised of a partially mirrored glass, plastic or film material. The two way 
mirror may be laminated with one or more sheets of glass or plastic to maintain a flat 
surface. The presentation screen can be a rear projection screen, a front projection 
screen, a retroreflective surface or a large display screen, such as a plasma screen, LCD 
screen or monitor. 

In the location of the origination of the reality interface conference, the home location, 
a user seated at a table looks forward to see the life-size image of a person from another 
location called the remote location. A two way mirror positioned in front of the user is 
angled to reflect an image of a person displayed on a screen located above the user. 
The screen and the mirror are configured so that the reflected image of the person on 
the screen appears to be seated in a chair on the other side of the table. The chair has a 
high back with a camera facing towards the user to capture the image of the user in the 
home location. The position of the height of the camera is set to match the position of 
the height of the eyes of the person in the remote location. 

The remote location has a similar configuration of the invention to display the image of 
the person in the home location and to transmit an image of the user in the remote 
location. 

The user in the home location sees the other half of the conference room through the 
two way mirror and sees an image of the distant user superimposed into a position 




behind the table in the plane that is in front of the chair. The camera on the user sees 
through the two way mirror to capture an image of the user. The video or computer 
image has the background dropped out to black so that the only image being displayed 
on the screen in the remote location is the image of the user. The background can be 
dropped out to black by lowering the lighting in the room behind the user or by using a 
chromakey technique to replace a colour, such as blue, with a black background. 

The invention can be configured to have more than two locations participating in the 
reality interface conference with each location having separate display screens for each 
person transmitted from the remote locations. In these configurations a separate 
camera is used for each view of the user. 

In another embodiment of the invention there are two cameras with an offset of 
approximately 65 mm to capture a stereoscopic image which is transmitted to the 
remote location or locations. The two cameras are positioned above the two way 
mirror so that their field of view covers a reflection of the user in the two way mirror. 
The exact location of the two cameras is determined by the position of the eyes of the 
user in the remote location. A head tracking system determines the exact position of 
the eyes of the user in the remote location and the co-ordinates are transmitted to the 
home location. In the home location the cameras are moved to the selected position 
above the mirror to match the relative position of the remote user behind the two way 
mirror. 

In this configuration the screen is comprised of retroreflective material and the images 
are front projected. The two projectors use single lenses and the projectors are 
positioned so that the focal points of the reflected images match to the positions of the 
two eyes of the user. The co-ordinates produced by the head tracking device are used 
to position the projectors in the remote location. The projectors are held in a translation 
stage that moves to position to be at the reflected focal point from the retroreflective 
surface. In this way the user will see an autostereoscopic image. 



In the home location the user will see an autostereoscopic image of the users in the 
remote location. This configuration does pose a problem in that the field of view of the 
cameras will include the projectors seen through the two way mirror. In order to 
minimise the brightness of these projectors it is possible to use polarising filters on the 
projectors and a second set of polarising filters set at 90 degrees on the cameras. 

Specific embodiments of the invention will be described by way of example with 
reference to the accompanying drawings in which: 

Fig. 1 shows the invention with a user seated at a table with a two way mirror that 
reflects an image of a user in a remote location and provides a view through the mirror 
to a setting with a chair so that the image of the remote user is superimposed into the 
scene and a camera captures an image of the user through the two way mirror. 

Fig. 2 shows the invention in the configuration in the remote location with a mirror 
image of all of the same features of the home location. 

Fig. 3 shows the invention with a shorter person at the home location. 

Fig. 4 shows the invention in the configuration of the remote location with the camera 
on the user in a lower position in order to match the height of the eyes of the user in the 
home location. 

Fig. 5 shows an embodiment of the invention used for autostereoscopic viewing of a 
person in a remote location with a pair of projectors projecting an image on a 
retroreflective surface and a pair of cameras capturing the reflected image of the user. 

Fig. 6 shows the embodiment of the invention in the autostereoscopic configuration for 
the remote location. 



Fig. 7 shows the invention with the user in the home location with a closer viewing 
position with an adjusted position of the projectors to match the relative distance from 
the retroreflective screen. 

Fig. 8 shows the invention in the remote location with the camera on the user moved in 
closer to match the relative distance of the user in the home position to the user in the 
remote location. 

Fig. 9 shows an embodiment of the invention where the person in the home location is 
superimposed behind a lectern and is viewed by an audience. 

Fig. 10 shows another configuration of the invention in a lecture theatre setting where 
the two way mirror is angled down to reflect a screen in front of the lectem. 

Fig. 1 1 shows a configuration of the invention where the user in the view of a camera in 
the home location is behind a lectem and has a view of an audience in a remote 
location. 

Fig. 12 shows the invention in a configuration of an exhibition stand where group of 
standing people can view a superimposed image of a user from the home location. 

Fig. 13 shows a top view of the invention in the configuration for an exhibition stand 
where the camera angle of view captures an image of a group of people. 

Fig. 14 shows the top view of the invention at the home location where the user can see 
an image positioned at an angle of view that matches the angle of view of the camera in 
the remote location on the group of people looking into the exhibition stand. 

Fig. 15 shows a top view of the invention in a configuration at an exhibition stand with 
two cameras to cover a wider area showing a group of people. 
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Fig. 16 shows a top view of the invention in a configuration at the home location with 
two screens that match the cameras angles of view of the group at the remote location. 

Fig. 17 shows a top view of triangular table that represents the layout for a reality 
interface conference for three people. 

Fig. 18 shows the top view of the invention in a configuration for the home location for 
a three way reality interface conference. 

Fig. 19 shows the top view of the invention in a configuration of the first remote 
location for a three way reality interface conference. 

Fig. 20 shows the top view of the invention in a configuration of the second remote 
location for a three way reality interface conference. 



Referring to drawing Fig. 1 a user 1 is in the home location for a reality interface 
conference. The user 1 looks forward to see an image on a screen 3 that is reflected off 
a two way mirror 2. The two way mirror 2 and the screen 3 are aligned so that the 
reflected image is superimposed in front of chair 7 and behind table 8 at position 4. 
The rear projection screen 3 has an image projected from projector 10 that is reflected 
off mirror 9. The user 1 can input computer commands through a keyboard in a 
console 6 which also incorporates a standard monitor or touch screen monitor. A 
camera 5 captures an image of the user 1 through the two way mirror 2. 

In Fig. 2 a user in a remote location 21 looks through a two way mirror 22 to see a 
reflected image of screen 23 that appears to be positioned in front of chair 27 and 
behind table 28 at position 24. The image at position 24 is the user 1 in Fig. 1 as seen 
by camera 5. The user 21 can input information and control computer functions at 
console 26 that will be linked by a network to the console 6 at the home location 
represented in Fig. 1 . A camera 25 captures an image through the two way mirror 22 of 
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the user 21 so that it can be transmitted to the home location for display on screen 3 in 
Fig. 1. 

Fig, 3 represents the home location with a different user 12 that is shorter than the 
original user 1 in Fig. 1. 

Fig. 4 shows the remote location with the same user 21 as illustrated in Fig. 2. The 
camera 25 is lower than in Fig. 2 so that the height of the camera 25 matches the 
position of the eyes of the shorter user 12 in Fig. 3. The output of the camera 25 is 
transmitted over a network to the home location and displayed on screen 3 in Fig. 3 so 
that the user 12 will see the reflected image in the position 4 at an angle of view that 
matches the angle of view of the user 12. 

Fig. 5 shows a user 31 in a home location for the invention in a configuration for an 
autostereoscopic reality interface conference. The user 3 1 looks forward through a two 
way mirror 32 to see a retroreflective screen 34. A projection rig 35 holds a pair of 
projectors that have projection paths that have a convergence that is approximately 65 
mm horizontally offset The rig with two projectors is positioned so that the projection 
reflects off the two way mirror 32 to the retroreflective screen. The light reflecting off 
the retroreflective screen passes through the two way mirror and is focussed onto the 
left and right eyes respectively of the user 31. A camera rig 33 holds two cameras so 
that the field^of view for the two cameras are offset approximately 65 mm in order that 
they can capture stereoscopic pairs of images. The cameras are directed toward the two 
way miiror at an angle to capture a stereoscopic image of the user 31. The user 31 
wears a head tracking device 36 that is used to provide co-ordinates of the position of 
the eyes of the user 31. The projection rig 35 is held in position by a translation stage 
37 that moves to projection rig 35 into a position matching the relative position of the 
eyes of the user 31. 

Fig. 6 shows a user 41 in a remote location for an autostereoscopic reality interface 
conference. The user 41 looks through the two way mirror to see the retroreflective 
screen 44. The projection rig 45 has two projectors that project the images captured by 
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the camera rig 33 in fig. 5. The camera rig 43 is positioned so that the field of view 
captures the image of the user 41 as seen as a reflection off the two way mirror 42. The 
images from the two cameras in the camera rig 43 are transmitted to the home location 
illustrated in Fig. 5 and projected by the pair of projectors in the rig 35 so that the user 
31 can see the stereoscopic image on the retroreflective screen 34. 

Fig. 7 shows the user 31 in the home location as he/she leans forward to get a closer 
view of the images from the remote location as displayed on the retroreflective screen 
34. The head tracking device 36 determines the co-ordinates of the location of the eyes 
of the user 31 . The translation stage 37 moves the projection rig 35 closer to match the 
relative distance and angle of view of the user 31 . 

Fig. 8 shows the remote location with the translation stage 48 having moved the camera 
rig 43 in a closer position to match the relative position of the eyes of the user 31 in the 
home location shown if Fig. 7. 

Fig. 9 shows configuration of the invention as a reality interface conference where the 
remote location is an audience in a theatre. The audience looks through the two way 
mirror 65 to see the lectern 64 and the curtain 68 and sees a reflection of screen 63 
which appears superimposed into the stage setting at position 62. The camera 61 
captures an image of the audience through the two way mirror 65. 

Fig. 10 shows another configuration of the invention with two way mirror 65 angled 
down to reflect the image on the screen 63 so that it appears superimposed into the 
setting behind the lectern 64 and in front of the curtain 68 at position 62. 

Fig. 1 1 shows the home location for the reality interface conference with user 51 
standing behind a lectern 55. A camera 53 has a field of view that captures an image of 
the user as seen as a reflection off the two way mirror 52. The user looks forward at the 
same angle of view as the camera 61 in Fig. 9 so that the user sees the audience 
displayed on the monitor or screen 54 in the same orientation as the view of audience 
67 in the remote location. 
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Fig. 12 shows the invention in an exhibition stand with a person 69 who could be one of a group 
of people looking into the stand to see a user from the home location superimposed into the stand 
at position 62 behind lectern 64 and in front of a background 68. Camera 61 captures a view of 
the person or group as seen through the two way mirror 65. 

Fig. 13 shows a top view of the exhibition stand with a group of people 69 within the field of 
view 70 as seen by the camera 61 . 

Fig. 14 shows a top view of the home location with the user 51 looking at the monitor 
or screen 54 showing the field of view 71 at the same angle of view 70 as the camera 
61 in Fig. 13 in the remote location. 

Fig. 15 shows a top view of the remote location at an exhibition stand with camera 61 
capturing an image of a group of people 69 with a coverage of a set angle of view 70. 
A second camera 71 covers an angle of view 72 that captures images of more people 
73. 

Fig. 16 shows a user 5 1 at the home location with a field of view 71 being displayed on 
a monitor or screen 54 that matches the angle of view 70 of camera 61 in the remote 
location in Fig. 15. A second field of view 73 that matches the angle of view 72 of the 
camera 71 the remote location in Fig. 15 is displayed on the monitor or screen 75. 

Fig. 17 shows a top view of the configuration of a reality interface conference with 
three people where user 1 sees an image of a user 81 in one remote location and user 82 
from another remote location. In this configuration the users 81 and 82 will also be 
able to see and communicate with each other. 

Fig. 18 shows the user 1 in the home location with a camera 83 to capture his/her image 
the correct angle for viewing by user 81 and a camera 84 to capture his/her image for 
the correct viewing angle for user 82. 
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Fig. 19 shows the user 81 with camera 85 capturing his/her image from the correct 
angle for viewing by the remote user 82 and the camera 86 capturing his/her from the 
correct angle of view for the user user 1 in the home location. 

Fig. 20 shows the user 82 with camera 87 capturing his/her image at the correct angle 
for viewing for the user 81 and camera 88 capturing his/her image from the correct 
angle for the user 1 in the home position. 
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